Method of operation

The definition of the relevant information for the execution of the test is in principle already established in the test plan (e.g. functional and technical designs, requirements, use cases, user manuals, interview reports, prototype, and reference system). However, it is possible that, in respect of the exit information, changes have taken place. The test plan should then be amended and the identification of the information reviewed. Finally, the various parts of the test basis are actually collected. Eventually, of course, the test team should have the correct (version of the) test basis at its disposal.

A point to bear in mind here is that the test basis does not always have to be present, complete, up to date, or established in documentation. A test basis often appears to be incomplete because, for example, non-functional requirements have not been specifi ed, while they are nevertheless considered to be risk-related. By alerting the project to this, a (timely) trigger is created for bringing it to attention.

Alternative test basis

If test basis problems do indeed arise, some solutions obtained from practice for obtaining an alternative test basis are listed below:

• Present system in production as reference system - Supposing the system documentation is missing, obsolete or incomplete in a conversion or migration project, for example. The creating, supplementing or updating of this documentation normally does not belong within the scope of the project. In such a situation, the present production version of the system is used as test basis. This is a particularly good alternative in situations that involve few or no changes to the functional operation of the system, or if the changes are well documented.
• Prototype as test basis - In a situation that does not accord high priority to the production of system documents, which are possibly only to be delivered at the end of the project, a prototype is sometimes made. This occurs, for example, with Rapid Application Development or variant of this (including SP, DSDM and RUP). Since the prototype is often made in co-operation with the user, this can also be used as the test basis.
• Information session - During, for example, maintenance operations, it often appears that neither the system in production nor the changes to it have been well documented. The organisation of information sessions for everyone involved (developers, designers, users, administrators, etc.) is a good way of clarifying both the operation of particular system parts and the changes to be implemented. The information obtained during such a session can be used as a test basis.
• System documentation from the last-but-one iteration as a test basis - With iterative and incremental system development approaches, there is a possibility that the system documentation will only become available to the tester at a later stage. In a situation where it is not permissible to change the system documentation during the last iteration, the test basis is made available to the tester at the end of the last-but-one iteration. In the situation where it is permissible to change the system documentation during the last iteration, it may be considered whether to use the system documentation from the last-but-one iteration as the test basis (often more than 80% ready). At the end of the last iteration, the – often small – changes to the system documentation have to be processed in the test cases by the tester.

An important point in connection with the above means of obtaining an alternative test basis is that this is seen by the client (and any other stakeholders) as the test basis. However, a test basis obtained in this manner will seldom be approved or consolidated. It is therefore important for the client and the tester to be aware of the risks that this involves. It is advisable not only to inventory these risks, but also to establish the associated countermeasures. For example, who has the ‘deciding vote’ if it appears that the realised functionality of a (sub)system differs from expectations based on the alternative test basis?

Occasionally, so little information is present that even establishing an alternative test basis is impossible. In such a situation, other sources of information may be resorted to, and while they cannot be used as an alternative test basis, they are perfectly usable for, for example, deriving logical test cases.

When using one or more of the approaches mentioned in Tips - Absence Of Test Basis with a view to arriving at an alternative test basis, or a basis for deriving test cases, the tester would do well to bear in mind that it is not the tester’s job to create the test basis. The tester assesses and uses the test basis exclusively for testing purposes. The creation of  system documents was, is and remains the responsibility of e.g. the project or the development department. The tester should avoid sitting in the place of the designer. This means that the test basis that is obtained from one of the above-mentioned approaches should always be agreed with all the stakeholders, on the one hand to confirm the way the system should function and/or be built, and on the other hand to confirm agreement that this is indeed the alternative test basis against which testing is to be carried out, or the basis from which test cases should be derived.

Products

Consolidated test basis

• HICCUPP [Bach, 2003]
• 18 Attacks by Whittaker and Jorgenson [Whittaker, 2000]
• Kaner’s 480 Bugs [Kaner, 1999].